Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

singular: update to 4.3.2, rebuild sagemath, networkx: update to 3.1 #43225

Merged
merged 3 commits into from
Apr 30, 2023

Conversation

tornaria
Copy link
Contributor

@tornaria tornaria commented Apr 2, 2023

Updates:

  • singular to 4.3.2
  • python3-networkx to 3.1

For these updates we need to rebuild sagemath with patches backported from development sagemath.

In addition we also add patches to sagemath to support the following pkg updates:

  • matplotlib 3.7
  • ipython 8.12
  • nauty 2.8.6
  • pythran 0.12.1

After this PR, sagemath testsuite should pass 100% with current versions of dependencies in void. All patches are taken from upstream.

Testing the changes

  • I tested the changes in this PR: briefly

I've been running this for a while. I'm also running sagemath 10.0.rc1 which includes all of our patches.

@tornaria tornaria marked this pull request as ready for review April 3, 2023 03:37
@tornaria
Copy link
Contributor Author

tornaria commented Apr 3, 2023

In addition to the checks here, I've run the long testsuite locally on x86_64 (CI runs only normal tests since running long tests takes twice the time).

Note this PR includes (a rebase of) PR #43153.

@tornaria
Copy link
Contributor Author

tornaria commented Apr 3, 2023

@dkwo

@dkwo
Copy link
Contributor

dkwo commented Apr 3, 2023

4.3.2p1 is also available, fix a few minor things

@tornaria
Copy link
Contributor Author

tornaria commented Apr 3, 2023

4.3.2p1 is also available, fix a few minor things

But it's not released, or is it? It's not in here: https://www.singular.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-3-2/

common/shlibs Show resolved Hide resolved
@dkwo
Copy link
Contributor

dkwo commented Apr 3, 2023

hmm, it's on github only, forget about that

@dkwo
Copy link
Contributor

dkwo commented Apr 3, 2023

then it looks good to me.
do you want me to test on aarch64?

@tornaria
Copy link
Contributor Author

tornaria commented Apr 3, 2023

do you want me to test on aarch64?

That would be great, and let me know anything weird. We could try to fix it in this PR.

Can you make a list of packages that are nocross? I know of ntl, fflas-ffpack, givaro, linbox, cysignals, sympow. It would be useful to fix those so they can be built and are available oob on void linux aarch64.

@dkwo
Copy link
Contributor

dkwo commented Apr 5, 2023

Besides the ones you mentioned, the list printed by xbps-src at the beginning includes

boost-devel *
openblas-devel *
singular *
eclib-devel
maxima-ecl
python3-Cython *
python3-cypari2
python3-fpylll
python3-pplpy
python3-primecountpy

(* means they are in this pr, or affected by big build in progress).
Moreover some of their dependants, like python-Cython, are being built.
I did not see givaro, as it is probably an indirect dependency that will be built by one of these.
For the tests, I have to wait until there's a big build in progress, otherwise it builds icu etc.

@dkwo
Copy link
Contributor

dkwo commented Apr 5, 2023

so basically fixing the ones you listed should fix everything :)

@dkwo
Copy link
Contributor

dkwo commented Apr 6, 2023

aarch64:

  • ntl, fflas-ffpack, givaro, linbox, cysignals, sympow, singular: all build and check fine
  • maxima: has emacs both in host and run deps, so it's not in the repo; must build with ~sbcl, otherwise it fails @paper42
  • sagemath: fails to start tests with aarch64-linux-gnu.so: undefined_symbol: fmpz_set_ZZ

@dkwo
Copy link
Contributor

dkwo commented Apr 6, 2023

(rebuilt and tested flintlib as well)

@tornaria
Copy link
Contributor Author

tornaria commented Apr 6, 2023

aarch64:
* maxima: has emacs both in host and run deps, so it's not in the repo; must build with ~sbcl, otherwise it fails @paper42

  • emacs is cross: we can add an emacs option and disable it for cross.
  • sbcl is also cross, so it should also be disabled at least for cross.

Do you mean maxima also fails when both sbcl and maxima are built natively?

* sagemath: fails to start tests with `aarch64-linux-gnu.so: undefined_symbol: fmpz_set_ZZ`

That is coming from flintlib. Are you using the cross built package or the native built one? The difference is that flintlib uses ntl if built natively and not if built cross.

What's the output of

$ objdump -T /usr/lib/libflint.so.17 | grep fmpz_set_ZZ
000000000009f8f0 g    DF .text	00000000000001ab  Base        fmpz_set_ZZ
000000000009faa0 g    DF .text	0000000000000005  Base        fmpz_set_ZZ_p

What about:

$ for p in /usr/lib/python3.11/site-packages/sage/libs/flint/*.so ; do echo $p ; readelf -d $p | grep libflint ; done
/usr/lib/python3.11/site-packages/sage/libs/flint/arith.so
 0x0000000000000001 (NEEDED)             Shared library: [libflint.so.17]
/usr/lib/python3.11/site-packages/sage/libs/flint/flint.so
 0x0000000000000001 (NEEDED)             Shared library: [libflint.so.17]
/usr/lib/python3.11/site-packages/sage/libs/flint/fmpz_poly.so
 0x0000000000000001 (NEEDED)             Shared library: [libflint.so.17]
/usr/lib/python3.11/site-packages/sage/libs/flint/ulong_extras.so
 0x0000000000000001 (NEEDED)             Shared library: [libflint.so.17]

@dkwo
Copy link
Contributor

dkwo commented Apr 6, 2023

yes, it first builds emacs, then sbcl, then gets stuck at some point in gdb.
i agree, we should add option emacs, and disable emacs,sbcl for cross.

i see, that makes sense for flint: my sage test was run before i rebuild flintlib. let me try again and report.

@tornaria
Copy link
Contributor Author

tornaria commented Apr 6, 2023

yes, it first builds emacs, then sbcl, then gets stuck at some point in gdb. i agree, we should add option emacs, and disable emacs,sbcl for cross.

I tried to cross build maxima with ecl, but it doesn't seem to work. The build finishes if I place ecl in hostmakedepends but then the binaries produced are x86_64 and not cross... Maybe it's possible to cross build with clisp, but that won't be enough for sage (we really need maxima-ecl).

What we could do is add the emacs option and set the default for aarch64 to ~emacs,~sbcl so you still have to build natively but at least you are not forced to build emacs and sbcl...

i see, that makes sense for flint: my sage test was run before i rebuild flintlib. let me try again and report.

@dkwo
Copy link
Contributor

dkwo commented Apr 6, 2023

I agree about maxima. For tests (with check=yes):

sage/graphs/generators/classical_geometries.py  # Killed due to segmentation fault
sage/graphs/strongly_regular_db.pyx  # Killed due to segmentation fault
sage/parallel/map_reduce.py  # Timed out
sage/sets/recursively_enumerated_set.pyx  #Timed out

@tornaria tornaria force-pushed the sagemath branch 2 times, most recently from c4789f7 to 046c53e Compare April 25, 2023 12:05
@tornaria
Copy link
Contributor Author

I did the following modifications to sagemath:

  • added a patch to fix doctests when pythran is installed (optional dependency)
  • rebased on top of python3-Babel: update to 2.12.1. #43497 to fix doctests when python3-Sphinx is installed (optional dependency)
  • added pythran and python3-Sphinx to checkdepends so doctests that depend on them are run

@dkwo
Copy link
Contributor

dkwo commented Apr 27, 2023

looks good to me.
have all our patches been merged upstream at this point?

@tornaria
Copy link
Contributor Author

Tests fail because python3-Babel is not rebuilt, even if I included #43497 here.

@tornaria
Copy link
Contributor Author

looks good to me. have all our patches been merged upstream at this point?

Yes, everything is included in 10.0.rc0, see: https://github.com/tornaria/void-packages/tree/sagemath-next if you want to try it (should I make a draft PR?)

@dkwo
Copy link
Contributor

dkwo commented Apr 28, 2023

should I make a draft PR?

that'd be great

Also fix doctests for pkg updates:
 - matplotlib 3.7
 - ipython 8.12
 - nauty 2.8.6
 - pythran 0.12.1

And add support for:
 - networkx 3.1

We use patches from upstream sagemath to fix all of these.

We also add pythran and sphinx to checkdepends to make sure doctests
pass when they are installed.
@tornaria
Copy link
Contributor Author

Here I added support for networkx 3.1 using the patch from sagemath/sage#35584.

For testing, I include a commit to update networkx to 3.1. This one can be merged independently (but only after the patch for sagemath to support networkx 3.1 is merged).

@icp1994

@icp1994
Copy link
Contributor

icp1994 commented Apr 29, 2023

Nice, thanks!

@tornaria
Copy link
Contributor Author

Nice, thanks!

Could you please try this branch and build the update of networkx to 3.1 to see if it's working ok for your purposes?

Maybe we can just merge this PR altogether as it is now. Otherwise I will try to get this merged without the networkx update and move the networkx update to a separate PR.

@icp1994
Copy link
Contributor

icp1994 commented Apr 30, 2023

Yes, I have been already using practically the same patch as this branch for my local use of networkx since v3 came out.

@tornaria tornaria changed the title singular: update to 4.3.2, rebuild sagemath singular: update to 4.3.2, rebuild sagemath, networkx: update to 3.1 Apr 30, 2023
@tornaria
Copy link
Contributor Author

@leahneukirchen could we get this merged please?

This brings sagemath up to date with current versions of packages in void; it also updates networkx which was requested by @icp1994 but cannot be updated without the patch I added here.

@leahneukirchen leahneukirchen merged commit 26d9c19 into void-linux:master Apr 30, 2023
@tornaria tornaria deleted the sagemath branch April 30, 2023 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants